Construct Validity and the O*NET Holistic Rating Scales: Evidence of a Fundamental Lack of Discriminant Validity

Positive claims regarding the construct validity of the Occupational Information Network (O*NET) have been made for years; however, reports of high scale multicollinearity (e.g., LaPolice, Carter, & Johnson, 2008) raise doubts regarding discriminant validity. I factor analyzed four O*NET surveys; because each single-item scale is meant to define a distinct construct, evidence of construct validity would be seen in high-dimensionality factor solutions and low communalities. Results revealed a fundamental lack of discriminant validity; instead, findings were consistent with the view that O*NET raters exhibit halo error, and rate based on their global views of the Data, People, and Things dimensions of Fine's Functional Job Analysis theory. Although O*NET's constructs may be conceptually distinct in the abstract, raters are decidedly unable to make empirically distinct ratings of them using O*NET scales; these findings further question the practice of using single-item holistic rating scales to directly rate hypothetical, unobservable, latent trait constructs.